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What’s Changed Since Apollo? 
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New Ways of Exploring 


Part 1 : Robots for human exploration 

• Improve planning for crew missions 

• Off-load “unproductive” tasks 

• Before, during, & after 

Part 2: Automated planetary mapping 

• Image base maps 

• 3D terrain reconstruction (DEM’s) 

• Very rapid updates 


Part 3: Participatory exploration 

• Public involvement in missions 

• Neo-geography & Web 2.0 tools 

• Citizen science & education 
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Part 1 : Robots for Human Exploration 


Purpose 

• Improve mission planning and 
crew productivity 

• Off-load “unproductive” tasks 
(tedious, repetitive, long-duration) 

Before crew 

• Recon (scouting) & prospecting 

• Site prep, deploy equipment, etc. 

Supporting crew 

• Inspection, mobile camera, etc. 

• Heavy transport & mobility 

After crew 

• Follow-up & close-out work 

• Site survey, supplementary tasks, etc. 
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NASA Human-Robotic Systems Project 


Research areas 

Surface mobility 

■ Crew 

■ Habitat 

■ Robots 
Handling 

■ Cargo 

■ Payloads 

■ Resources 

Human-robot 
interaction (HRI) 



Primary objectives 

• Address key technical challenges for lunar surface operations 

• Develop requirements & mature systems for lunar surface operations 

• Perform trade studies in laboratories and analog environments 


NASA Centers: ARC, GRC, GSFC, JPL, JSC, KSC, LaRC 
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HRS Field Testing 


Test and validate 

• Technologies, systems, & protocols 

• Integrated mission simulations 


Analogs are never 100% perfect 

• No place on Earth is exactly the Moon 

• Many factors to consider 


Factors and fidelity 


Applications 

Science 

Aspects 

Exploration 

Climate 



Geology 


Learn Test Train 

Functions 

Biology 

Engage 


Terrain 



Fidelity 

Low Medium High 

Logistics 

N/A 

Other 


Value 


Applications Functions Aspects 

Fidelity 

Cost-Effectiveness 
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Black Point Lava Flow 
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2006 Meteor Crater Field Test 


3-16 September 2006 

• Coordinated human-robot operations 

• ARC, JSC, JPL, & LaRC 

• Co-located with Desert RATS (shared infrastructure) 
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Lunar Short Stay Mission Simulation 


[T ATHLETE positions 
Pressurized Rover 
Compartment (PRC) 

[2] Crew drive unpressurized 
rover to worksite 




Crew dismount and walk to 
PRC to recharge suits 


[4] Centaur removes sample 
box (time-delayed teleop 
via satellite from Houston) 
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K10 performs autonomous 
“walkaround” (for remote 
visual inspection) 
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Visual Inspection 



M. Bualat, L. Edwards, et al. (2007) 
“Autonomous robotic inspection for 
lunar surface operations” 
Field & Service Robotics 


Robot-based imaging 

• Autonomous approach & 
inspection photography 

• HDR gigapixel panorama 

• Crew (IVA or ground) analyzes 
images for problems 
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Basic Panorama 
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HDR Panorama 




2007 Haughton Crater Field Test 


10 July - 3 August 2007 

• Systematic site survey with two K10 robots 

■ 3D scanning lidar for topographic mapping 

■ Ground-penetrating radar for resource prospecting 

• Multiple lunar analog sites at Haughton Crater 

• Remote (habitat & ground control) robot operations 



Haughton Crater 
(Devon Island, Canada) 



T. Fong, M. Allan, et al. (2008) 
“Robotic site survey at Haughton Crater” 
Inti. Symposium on Al, Robotics, & Automation in Space 
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Haughton Crater 
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• Devon Island: 66,800 sq. km 

• Largest uninhabited island on Earth 

• Haughton Crater: ~20 km (diameter), ~39 Ma (Late Eocene) 
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Haughton Crater 
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2007 Haughton Crater Field Test 






3D Mapping (Terrain 



K10 Red 
lidar survey 
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“Fortress” formation 
near HMP base camp 





3D Mapping (Terrai 
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elevation map 



3D Mapping (Structures) 



(1m polar grid) 
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“Drill Hill” Subsurface Survey 



Ground Penetrating Radar survey 

• Resource prospecting: subsurface scans for buried water ice 

• Parallel transects (lawnmower pattern) with 50 m spacing, E-W & N-S 

• Survey speed limited by sensor data acquisition rate ! 
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“Drill Hill” Survey 



Survey plan \ 
(green) 


Survey boundary 

(blue) 


Parallel line transects 
(50 m spacing, E-W, N-S) 


K10 path 
(black) 


20.5 km total drive 
82,000 measurements 
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2009 Robotic Recon Experiment 


Focus 

• Test robotic recon in advance 
of crew (LER-based mission) 

• Test coordinated human-robot 
field exploration technique 

• Improve productivity & science 
during human exploration missions 



Objectives 


robot ■ ■ ■ crew ■ ■ ■ 


• Assess effect of robotic recon on 
traverse planning & crew productivity 

• Capture requirements (instruments, 
comm, nav, etc.) for robotic recon 

• Provide recommendations to lunar 
architecture and exploration planning 



M. Deans, T. Fong, et al. (2009) 
“Robotic scouting for human exploration” 
AIAA Space (AIAA-2009-6781 ) 
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Why Is Recon Useful? 




SOI ITH MASSIF 
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2009 Robotic Recon Experiment 


Pre-Recon 


Robotic Recon 
Mission 





Mar 1 - June 1 

Satellite images 
(LRO equiv.) 

Geologic map 


June 14 - 26 

• KIOatBPLF 

• Ground control 
at NLSI 



July 1 - Aug 15 

• Satellite images 

• K10 recon data 
(images, lidar) 


Aug 29 - Sep 3 

• LERatBPLF 

• Science backroom 
at BPLF 
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Lunar Analog Site 


Black Point Lava Flow 

• 65 km N of Flagstaff, AZ 

• Analog of the “Straight Wall” 
(Mare Nubrium / Rupes Recta) 

• Basaltic volcanic rocks & 
unit contacts 
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Black Point Lava Flow 




Preliminary Crew Traverses 
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Robotic Recon Mission (June 2009) 


VIDEO 


tk-i' a 


lunarscience . nasa . gov/roboticrecon 
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Robotic Recon Instruments 



3D scanning LIDAR 


• 3D topography 
measurements 

• 5mm @ 500m 

• Oblique views not 
possible from orbit 
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GigaPan 


• Oblique, wide-angle, 
color, context views 

• 60x180 deg 

• >100x resolution of 
LRO LROC-NA 



Microscopic Imager 

• High-res, close-up, 
color, terrain views 

• 72 micron / pixel 

• >7,000x resolution of 
LRO LROC-NA 
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Collected Recon Data 
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8.5 GB data collected (52 hrs of robotic recon operations) 

39 LIDAR scans, 75 GigaPan, and 95 terrain images 
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Collected Recon Data 







Orbital Data 



Digital Globe QuickBird (60 cm/pixel) 
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Surface Data 




GigaPan panorama close-up 
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Terrain image (55 microns / pixel) 
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Crew Mission (September 2009) 


Lunar Electric Rover (LER) 

• Prototype pressurized crew 
vehicle for lunar operations 

• Two “suit ports” for rapid (15 min) 
egress and ingress 

• 20 km/hr max, active suspension 

• 3.5 x 5 m (wheelbase x length) 

Crew A 



• Mike Gernhardt & Brent Garry 

• W1 (pre-recon) + N2 (post-recon) 
traverses 

Crew B 

• Andy Thomas & Jake Bleacher 

• N1 (pre-recon) + W2 (post-recon) 
traverses 
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Crew Mission (September 2009) 
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Impact on Traverse 


“West” region 

• W1 (pre-recon) plan was 
Apollo-like exploration 

■ Rapid area coverage 
(visit 5 geologic units) 

■ Assumed single visit 

• W2 (post-recon) plan was 
significantly different than W1 

■ Improved target prioritization 

■ Recon data supported 
real-time replanning 


T. Fong, A. Abercromby, et al. (2009) 

“Assessment of robotic recon for 
human exploration of the Moon” 
International Astronautical Congress 
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Interactive Planetary Robotics 


K10 Robot 


£ yl - 
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Lunar Analog (Black Point Lava Flow) 





Science Operations Team 


Scientists 



Robot Team 


Experiment 


Liason 


PI 


Robot PI 


Flight Control Team 

Flight Science 




Data 

Curation 


Ground Data System 
(COTS based) 


Ground control (NASA Lunar Science Institute) 
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Ops Drivers 
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Ground Control Structure 
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T. Fong, M. Bualat, et al. (2008) 
“Field testing of utility robots for lunar surface operations” 

AIAA Space (AIAA-2008-7886) 
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K10 Ground Data System 








Google Earth Traverse Planning 
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Google Earth Monitoring & Data Viz 
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In-Line Performance Metrics 


Continuous robot monitoring 

• Monitor robot telemetry 

• Compute task & system metrics 

• Web-based displays for flight control 

Metrics for flight control 

• Plan timers (planned, elapsed, etc.) 

• Instrument timers (data acquisition) 

• Loss of Signal 

Metrics for shift debrief 

• Productivity (WEI, % time on task) 

• Reliability (MTTI, MTBI, interventions) 

• Loss of Signal 




D. Schreckenghost, T. Fong, et al. (2009) 
“Measuring robot performance in real-time for NASA robotic recon operations” 

Performance Metrics for Intelligent Systems Workshop 
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Performance Displays (Shift Debrief) 


Debrief History CZ 

4 - C # _j file:///U sers/schreck/Documents/hri-sbir/ph2-sbir/Jun09FieldTest/BlackPoint-6-09/ops/p ^ T 1 “ 


Debrief Summary 
for each day 


Most Visited 

^ Getting Started 

Latest Headlines ^ 20090624 Houston TranStar Re. . 


r □ 

Debrief History 

+ 



^Debrief History 
15Jun PM 
■ Si 16Jun AM 
Si 16Jun PM 
Si 16Jun PM (Restart) 
Si 17Jun PM 
Si 18Jun Full Day 

il 1ft Inn D M fReSta 


Ratio: Productive Time 
to Overhead Time 


Si 23Jun PM (Restart) 
Si 25Jun Full Day 
Si 26Jun Full Day 


Performance Summary 

Elapsed Time: 04:36:17 


Thu Jun 25 2009 AM 

Show Anomalies 






Task breakout 
for day 


Productive Time 02:55:29 
Overhead Time 01 :36:54 
WEI 1.81 


pw»0(i 


Drive 00:41:05 2417.66 

Lidar 00:18:22 0 P; 3 S 

PanCam 01:42:06 800 

Microimage 00:02:49 12 


1 3:50:50 Plan BPA_B054D_PLAN uplinked 
13:50:50 Plan execution goes idle. 

13:50:50 New plan uploaded. 

1 3:51 :1 3 Rover will traverse in blind mode for this plan 
13:51:24 Execute 54D 

13:51 :24 Plan is executing. 

1 3:54:51 ROC: request laptop battery change when power up lidar 
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Daily log interleaves 
automatic events with 
human logs 
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Part 2: Automated Planetary Mapping 


Purpose 

• High-quality digital maps 

• On-line access 

• Very rapid updates 

Data processing 

• Orbital imagers (Moon, Mars, etc.) 

• Image base maps 

• 3D terrain reconstruction (DEM’s) 

Data fusion & retrieval 

• OGC standards (WMS, WFS) 

• Geobrowser markup (KML, WTML) 

• Image metadata (GeoTIFF, etc.) 
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Planetary Data Firehose 
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MGS Odyssey MER MRO Apollo SELENE ChangE-1 


Source: 


B. Archinal, L. Gaddis, et al. (2007) 

“Urgent Processing and Geodetic Control of Lunar Data” 

Workshop on Science Associated with the Lunar Exploration Architecture 


Chandra 

yaan-1 


LRO 
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Planetary Data Firehose 


Traditional mapping 

• Human-intensive 
cartography 

• Manual control & 
error analysis 

• Maps take years 
to complete 

Image resolution 

• Imagers keep 
getting better 

• High-res digital 
scans of old film 
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Apollo Metric 
Camera 

16,000 x 16,000 


HRSC 


CTX 


5184 x 
16000 


5064 x 
16000 


MOC-NA 

2048 x 4800 

MER Pancam □ 


HiRISE 

20,000 x 40,000 


1024 x 1024 


Not shown: Apollo Panoramic Camera 

25400 x 244000 


LROC 

10000x50000 
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Automated Stereo Image Processing 
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Automated Stereo Image Processing 


Problem 


• Multiple images from 
different viewing angles 

• Compute 3D terrain model 
with no human intervention 



Open-source C++ packages 

■ NASA Ames Stereo Pipeline 

■ NASA NeoGeography Toolkit 

■ NASA Vision Workbench 
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M. Broxton and L. Edwards (2008) 
“The Ames Stereo Pipeline: Automated 3D surface reconstruction from orbital imagery” 

Lunar and Planetary Science Conference 
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Mars Terrain Models 



MGS MOC Narrow Angle 

• Collaboration with Malin Space 
Science Systems 

• Adapted Ames Stereo Pipeline to 
orbital images 


MRO Context Imager (CTX) 

• Collaboration with CTX Team 

• Provided rapid turn-around 
stereo modeling 


Mars Express HRSC 

• Collaboration with USGS, DLR 

• Formal comparison of Digital 
Elevation Model (DEM) products 

• Four controlled data sets 
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Mars Terrain Models 
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Mars Terrain 



NE Terra Meridian i (2.38° N, 6.4°E) 
(from MOC E04-01109 + M20-01357) 
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Mars Terrain Models 
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Lunar Terrain Models 



Apollo Panoramic Camera 

• 3D terrain model of 
Taurus-Littrow valley 
(Apollo 17) 

• Featured at the Hayden 
Planetarium (American 
Museum of Nat. History) 


Apollo Metric Camera 


• Systematic creation of 
image maps & DEMs 

• Refinement of the Lunar 
geodetic control network 
(with USGS) 




NASA Lunar Mapping & 
Modeling Project 
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Lunar Terrain Mode 


Apollo Image Archive project 

• apollo.sese.asu.edu 

(Mark Robinson, ASU) 

• Photogrammetric scans of 
original Apollo films 

(200 pixel/mm, 14-bit) 
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Lunar Terrain Models 


Large-scale mapping 

• Systematic reconstruction of the “Apollo Zone” 

• Stereo vision & mosaicking of Apollo Metric Camera scans 

• 8,000 stereo pairs from Apollo 15-17 





Planetary exploration REBOOTED ! 


55 



Prior Work in Mosaicking 


CTX polar mosaic (2007) 

• 40.3 gigapixel mosaic 
(610 thousand tiles) 

• 305-GB of source imagery 

• 5 CPU-days of processing 

• Collaboration with 

Mai in Space Science Systems 

Old mosaic engine 

• Single-threaded 

• One-shot (if it crashed, 
had to start over...) 

• Could not incrementally 
add imagery to a mosaic 
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Improved Mosaic Engine 


Key features 

• High-performance, distributed processing (NASA Nebula cloud computer) 

• Supports incremental updates & roll-back on mosaicking failures 

• Aggregation of image tiles into indexed “plate files” 

(highly scaleable and efficient) 
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Improved Mosaic Engine 
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Apollo 15 Orbit 33 
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M. Broxton, A. Nefian, et al. (2009) 
”3D Lunar terrain reconstruction from Apollo Images” 
Int’l Symposium on Visual Computing 

0C7 



Coming Soon ... 


Complete HiRISE Mosaic 

• Mars Reconaissance Orbiter HiRISE imager 

• Each image: 20,000 x 50,000 pixels 

Mosaic stats 


Tile Dimensions 

256 x 256 pixels 

Root Tiles / Image 

15,000 

Tile Space 

25 KB 

Tiles Total 

229 million 

Total Mosaic Size 

5.7 TB 
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Part 3: Participatory Exploration 


Purpose 

• Enable everyone to actively 
participate in NASA missions 

• Explore space in bold, new ways 

• Engage & educate students 



NASA data for everyone 

• Easy access to planetary data 

• Reach millions of users 
(very low barrier to entry) 

• Neo-geography browsers 

(Google Earth, WorldWideTelescope) 

Citizen science 

• Volunteers help perform science 

• Informal & formal education 

• Social networking 
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NASA Data for 


Our goals 

• Make NASA’s geospatial data 
universally and easily accessible 

• Enable millions of people to find 
and use NASA data 

• Improve planetary science & 
exploration missions 

How do we do this? 

• Process raw data into maps 
(mosaics, terrain models, etc) 

• Support geo-browsers & GIS 
platforms through open standards 

• Provide incremental updates, so that 
data can be shared in near real-time 
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UniView 




Mars in Google Earth 


a.k.a. “Google Mars 3D” 

• Launched Feb. 2, 2009 

• Co-developed with Google 

• Built in to Google Earth v5 

Content 

• Global maps: topography, 
infrared, historical, etc. 

• Imager footprints & overlays: 
HiRISE, CTX, MOC, etc. 

• MER tracks & panoramas 

• Tours (Bill Nye & Ira Flatow) 

• Live from Mars: THEMIS 
images within hours 

• And much more ... 



R. Beyer, M. Broxton, et al. (2009) 
’’Visualizing Mars data and imagery with Google Earth” 

American Geophysical Union 
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Moon in Google Earth 



a.k.a. “Google Moon” 

• Launched July 20, 2009 

• Co-developed with Google 

• Built in to Google Earth v5 

Content 

• Global maps: topography, 
geologic, historical, etc. 

• Spacecraft imagery: Apollo, 
Clementine, Lunar Orbiter 

• 3D models of spacecraft, 
landers, and crew rovers. 

• Tours (Andy Chaikin, Buzz 
Aldrin & Jack Schmidt) 

• And much more ... 


M. Weiss-Malik, T. Scharff, et al. (2009) 
’’Visualizing Moon data and imagery with Google Earth” 

American Geophysical Union 

Planetary exploration REBOOTED ! 
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GigaPan 


Explorable images to engage, inform, and inspire 

• Interactive, gigapixel panoramas (gigapan.org) 

• Robotic pan/tilt head + stitching software + collaboration web site 

• Co-developed with CMU 




GigaPan on the Lunar Electric Rover 


Desert RATS 2009 

• GigaPan “Voyage” (rugged pan-tilt + embedded processing/server) 

• Remotely operated by science backroom (ground control) 

• Context & high-resolution imaging for field geology 



| Traverse A/7, Station 1 (Aug 29, 2009) 

290x40 deg panorama (112 images) 
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Participatory Explorath 


Objectives 

• Citizen science using GigaPan 

• Involve public & students 

Pre-test (Dec 2009 - May 2010) B 

• Manually take GigaPans on 
preliminary LER traverse plan 

• Public explores panoramas, takes 
snapshots & adds comments 

• On-line discussion w/ scientists 
(places to stop, tasks, etc.) 

• Public input for final plan 

Field-test (Aug - Sept 2010) B 

• Take GigaPan panoramas from 
LER & publish via gigapan.org 

• Public explores panoramas & 
discusses discoveries w/ scientists 
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for D-RATS 2010 
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Citizen Science 
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Citizen Science 


Formal education (pre-test) 

• Higher education (college students) 

• Structured classroom activities 
(e.g., geomorphology) 

• Collaboration: Kip Hodges (ASU) 

Informal education (field-test) 

• K-1 2 students 

• Museum summer programs 

(e.g., San Francisco Exploratorium) 

• Collaboration: lllah Nourbakhsh (CMU) 


“Web 2.0” engagement 

• General public 

• “Viral” involvement via social 
networking & modern web platforms 

• Collaboration: Google & Facebook 
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GigaPan in Google Earth 



Courthouse Butte, Sedona, AZ 
(60 cm/pixel satellite image on 10 m/post DEM) 
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GigaPan in Google Earth 
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GigaPan in Google Earth 
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GigaPan in Google E; 
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Conclusion 


New ways to explore 

• Robots for human explorers 

• Automated planetary mapping 

• Participatory exploration 


Asteroids & 
Near-Earth Objects 




Deep 

Space 
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International Space Station 




Lunar Orbit, 
Lunar Surface (Global) 
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NASA Summer 2010 Internships 


Undergraduate Student Research Project (USRP) 

• Sophomores, Juniors, & Seniors 

• Deadline: January 22, 2010 

• http://www.epo.usra.edu/usrp 

Motivating Undergraduates in Science and Technology (MUST) 

• Sophomores, Juniors, & Seniors 

• Deadline: February 1, 2010 

• http://scholarships.hispanicfund.org/applications 

Intelligent Robotics Group 

• Graduate students 

• Deadline: February 28, 2010 

• e-mail: terry.fong@nasa.gov 
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Questions? 



Intelligent Robotics Group 

Intelligent Systems Division 
NASA Ames Research Center 

irg.arc.nasa.gov 
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